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Summary 

With  disturbances  modeled  by  arbitrary  solutions  to  a linear  homogeneous 
differential  equation,  a deterministic  theory  is  developed  for  parameter  esti- 
mation using  data  over  a limited  time  interval  which  has  applications  to  sig- 
nal estimation  and  system  identification.  Linear  and  certain  classes  of 
nonlinear  and  time  varying  systems  can  be  treated  for  identification  purposes. 
The  approach  circumvents  the  need  to  estimate  unknown  initial  conditions 
through  the  use  of  a certain  projection  operator. 
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I.  INTRODUCTION 

The  success  in  using  stochastic  disturbance  and  measurement  noise  models 
with  underlying  Markov  process  representations  is  well  documented  for  signal 
estimation  and  system  identification  L 1 D . Yet  this  does  not  preclude  the 
possibility  that  alternative,  i.e.,  nonstochastic,  models  might  be  advan- 
tageous in  certain  situations.  This  paper  is  concerned  with  one  such  potentially 
useful  approach  in  which  the  disturbances  are  modeled  as  arbitrary  solutions 
to  a linear  homogeneous  differential  equation  of  pre-selected  order  (r)  over 
a fixed  finite  time  interval,  0 < t * t , and  "one  shot"  signal  estimation 
or  system  identification  is  to  be  undertaken  based  on  the  observed  data  on 
[O.tj]  . No  assumptions  are  made  regarding  either  the  coefficients  or  the 
initial  conditions  for  this  differential  equation,  so  that  a variety  of  dis- 
turbances on  [0,t1]  can  be  represented  by  a moderately  low  order  model, 
e.g.,  r = 3 or  4 , if  the  observation  time  interval  is  of  rather  limited 
duration. 

Although  such  disturbance  models  have  been  utilized  in  the  past  in  connec- 
tion with  compensator  design  for  the  servomechanism  problem,  [2]  and  [3],  or 
more  recently  by  Davison  [4]  for  a "compensator  identification"  problem  with 
asymptotic  tracking  properties,  the  use  of  this  model  is  quite  different  here. 

In  the  first  place,  the  time  interval  [O.t^]  is  finite  and,  at  least  theo- 
retically, may  be  arbitrarily  short.  Secondly,  no  restrictions  are  placed 
on  the  disturbance  modes,  i.e.,  they  may  be  stable,  unstable,  or  a mixture 
of  both,  finally,  no  attempt  is  made  to  identify  the  initial  conditions, 
either  in  the  system  model  or  in  the  disturbance  model.  This  latter  property 
results  from  the  application  of  an  annihilating  filter  (introduced  in  [5]), 
which  zeroes  the  initial  condition  response  of  a linear  system  on  a fixed  time 


t 


I 


the  class  of  models  to  include  certain  nonlinear  and  time-varying  systems 


allows  for  models  in  which  the  parameters  enter  nonlinearly,  and  includes  a 


signal  estimation  problem  as  an  application  of  the  basic  formulation.  In 


addition,  the  parameters  for  the  disturbance  model  are  represented  explicitly 


in  this  paper,  rather  than  implicitly  as  in  the  case  of  [5] 


The  theoretical  development  of  the  approach  is  given  in  Section  II 


starting  from  a basic  differential  operator  model.  Two  formulations  are 


presented  depending  on  whether  the  model  parameters  are  "separable"  or  not 


In  each  case  the  parameter  estimation  problem  is  transformed  into  a certain 


kind  of  least  squares  fit  with  the  pertinent  functional  obtained  as  an  inner 


product  over  a subspace  of  the  function  space  to  which  the  observed  data 


is  assumed  to  belong.  This  subspace  results  from  the  use  of  the  annihilating 


filter  and  serves  to  obviate  the  need  to  estimate  the  unknown  initial  conditions 


It  is  then  shown  in  Sections  III  and  IV  how  particular  identification  problems 


can  be  translated  into  the  basic  formulations  developed  in  Section  II 


Computational  considerations  are  briefly  discussed  in  Section  V,  but  actual 


numerical  results  are  reported  elsewhere 


II.  THEORETICAL  DEVELOPMENT 


It  is  shown  in  Sections  III  and  IV  how  particular  versions  of  the  signal 
estimation  and  system  identification  problems  can  be  viewed  as  finding  a 
parameter  vector  0 = (6^.0^)  which  satisfies  a differential  operator 
equation  of  the  generic  form 

P(D)v(t ) - Q(D)g(t  ,6)  = 0 , 0 < t t tx  (1) 


where  P and  Q are  polynomial  matrices  in  the  differential  operator 


given  by 

P(D)  = l PJ)0'1  , Q(D)  = l Q^Dn-1 

i=o  i=o 

and  (v(t) ,g(t,0))  are  column  vector  functions  of  the  given  data  on  [O.t^] 
and  parameter  vector  0 as  indicated.  With  respect  to  smoothness,  v(t)  and 
g(t,0)  are  assumed  to  be  piecewise  continuous  functions  of  t on  [O.t^]  and 
are  presumed  not  to  depend  on  derivatives  of  the  data;  g(t,0)  is  assumed  to 
be  continuously  differentiable  with  respect  to  0 for  each  fixed  t . 
Definition  The  basic  model  (1)  is  said  to  be  separable  in  the  parameters 
if  g(t  ,0 ) admits  to  the  representation 


g(t,0)  = V(t )f(0 ) 


(2) 


where  V(t)  is  a matrix  valued  function  of  the  data  and  f(*  ) is  a continu- 
ously differentiable  vector  valued  function  of  the  parameters  with  the  single 
valued  property: 

f ( 0 ) = f (0  ) if  and  only  if  0=6 

* 

for  all  0 and  0 


(3) 


A 


It  will  be  shown  that  the  computational  burden  is  significantly  less  for 
separable  models,  and  that  the  range  of  f ( • ) is  generally  of  higher  dimen- 
sion than  its  domain. 

Although  system  identification  will  be  discussed  more  fully  in  Section  IV, 
the  following  simple  example  will  illustrate  the  above  notation  before  con- 
tinuing with  the  development. 

Example  1 Consider  the  identification  of  parameters  in  the  Mathieu  equation 
with  disturbance  input  d(t)  : 

jKt)  + Ca1  - a2  cos  «3t]y(t)  = 6[u(t)  + d(t)]  , 0 £ t £ (4) 

based  on  the  observed  input-output  pair  [u(t),y(t)]  on  the  observation  inter- 
val [0,t^]  . Assume  the  disturbance  model 

d(t)  + u>d(t)  = (D  + <o)d(t)  = 0 , 0 ( t < t,  (5) 

where  to  is  a parameter  which  is  to  be  identified  along  with  the  other  system 
parameters.  The  disturbance  d(t)  can  be  eliminated  from  (4)  by  operating 
on  both  sides  with  (D  + to)  . After  rearranging  terms  the  resulting  differ- 
ential equation  can  be  expressed  as 


D3y(t)  + CD2  D 1] 


toy(t) 

[a1~a2cosa3t]y(t)  - 8u(t) 


= 0 


(6) 


|a>{[a1-a2cosa3t]y(t)  - 6u(t))J 

which  is  of  the  form  (1)  with  P(U)  = D3  , Q(D)  = -Row(D2,D,l)  and 
6 = (a  , a2,  <»3 , 8,  to)  . 

The  model  (6)  is  nonseparable  in  the  parameters;  however,  it  reduces  to 
the  separable  case  if  the  parameter  a3  is  a known  constant  since  the  g(t,0) 


vector  can  then  be  written  as  follows : 


-5- 


wy(t) 

[o^-o^cosagtlytt)  - Bu(t) 
u>{Co1-a2cosa3t]y(t)  - Bu(t)) 


y(t)  0 0 0 0 

0 y(t)  -y(t)coso3t  -u(t)  0 

0 0 0 0 y(t ) - 


where  V(t)  depends  on  the  given  input-output  data,  the  parameter  vector  is 
now  8 = (a^,  a^,  8,u>)  and  f(0)  satisfies  the  single-valued  condition  (3). 

Given  the  polynomial  matrix  pair  [P(D),Q(D)]  in  the  basic  model  (1),  let 
a square  polynomial  matrix  F(D)  be  selected  in  the  form 

m 

F(D)  = l FjD"1"1  , m i n (8) 

i=o 

with  the  integer  m and  coefficient  matrices  F^  chosen  so  that  F ^(D) 
exists  and  F 1(D)[P(D) ,Q(D)]  is  a causal,  i.e.,  proper,  transfer  function 
matrix.  Then  define  an  auxiliary  error  function  z(t)  = z(t,0)  implicitly 
through  the  solution  to  the  differential  operator  equation 


= V(t)f(«) 


0 

0 

y(t)cosa3t 


0 

0 

-u(t) 


B 

wa, 

d. 

w6 


(7) 


F(D)z(t)  = P(D)v(t ) - Q(D)g(t ,0 ) , 0 < t t t 


(9) 
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4- 

If  (A ,C ) is  an  observable  matrix  pair  for  the  homogeneous  part  of  (9)', 

then  the  solution  for  any  particular  initial  condition  x can  be  written  as 

o 

z(t,0)  = CeAtxo  + h(t)  - |i(t,e)  , 0 { t < tx  (10) 

where  h(t)  and  p(t,0)  are  the  zero  state  solutions  to 

F(D)h(t ) = P(D)v(t ) (11-a) 

and 

F(D)y(t,0)  = Q(D)g(t,0)  (11-b) 

respectively.  In  the  case  where  the  basic  model  (1)  is  separable  in  the 
parameters,  the  vector  p(t,0)  can  be  written  as 

u(t  ,0)  = M(t)f(0)  (12) 

where  M(t)  is  the  zero  state  solution  to  the  matrix  differential  equation 

F(D )M(t ) = Q(D )V(t ) . (13) 

Given  the  polynomial  matrix  F(D)  and  a corresponding  observable  matrix  pair 
(A,C)  for  the  homogeneous  part  of  the  auxiliary  error  equation  (9),  the  anni- 
hilation filter  relative  to  a fixed  observation  time  interval  [O.t^]  is 
defined  by  the  (noncausal)  kernel  function  matrix 

H ( t , t ) = I4(t-r)  - CeAtW_1eA'TC'  (i4-a) 

0 i t { t:  , 0<T$t1 

where  5(t)  is  the  Dirac  delta  function,  I is  the  identity  matrix,  W 1 is 


That  is,  all  solutions  to  F(D)z(t)  = 0 can  be  expressed  by 

z(t)  = Cx(t)  = Ce  xq  , x(t)  = Ax(t),  x(o)  = xot^n,  for  an  appropriate  observable 

• — A tr 

matrix  pair  (A,C)  with  minimal  dimension  state  space  (n)  , and  where  e is 

the  state  transition  matrix  for  A. 


_ 
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inverse  of  the  observability  Gramian  for  the  pair  (A,C)  defined  by 


W . [W 

* n 


„ At 
Ce  dt  , 


(14-b ) 


and  prime  denotes  transpose.  If  ^ denotes  the  Hilbert  space  of  all  vector 
valued  square  integrable  functions  z(t)  which  are  possible  solutions  to  (9) 
on  [0,t^]  , and  the  linear  subspace  spanned  by  the  columns  of  CeAX 
then  it  is  easy  to  show  that  $ is  a projection  operator  on  with  range 
contained  in  (^~^<>)  which  possesses  the  annihilation  property 


ii#(CeA*x  ) = 0 , 0 < t i t,  , for  all  x z.y0n 

o i o 


(15) 


This  property  follows  immediately  from  the  definition  of  & in  (14).  The 
fact  that  is  a projection  follows  upon  noting  that  &Q&)  = . 

Operating  on  both  sides  of  (10)  with  tit  yields 


z(t,G)  = h(t ) - u(t , 0)  , 0 < t < t1 


(16) 


where 


h(t)  = h(t)  - CeAVX 


eA'TC'h(T)dT 


(17-a) 


and 


p(t,6)  = u(t,6)  - CeAtW_1 


5A’TC'Vi(T,0)dT 


( 17-b) 


Thus,  O = (z(t)  = CeAtx  , 0 < t i t.  : x E^n) 

O X o 
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are  the  projections  of  h(*)  and  y(*,0)  dovm  into  the  subspace  (^-  ^-„)‘ 
Defining  the  inner  product  functional  J^Ce)  by 


J^e)  = <z(0),  z(6)  > 


f1- 

= z ' (t, 

* n 


e)z(t,e)dt 


it  follows  from  the  above  development  that  any  value  of  0 which  satisfies  the 
basic  differential  operator  model  (1)  is  also  a solution  to  the  nonlinear  trans- 


cendental equation 


J1(0)  = 0 . 


Conversely,  any  value  of  0 which  satisfies  (19)  is  a candidate  for  a value  of 
the  parameter  vector  0 satisfying  (1). 

A straightforward  calculation  of  the  quantities  involved^ shows  that  J1(0) 
can  be  represented  by 


f 1 -1  f 1 

J (0)  = h'(t)h(t)dt  - n'W  n - 2 h'(t)y(t,e)dt 

1 * 0 ' 0 

tl  (2C 

+ 2n'W_1v(0)  + y 1 (t ,0  )y(t  ,0  )dt  - v’(0)  w Xv(0) 

•*o 

where  h(t)  and  y(t,0)  are  given  by  (11)  and  the  vectors  (n,v(0)  ) are 


defined  by 


..fVv 

• n 


h(t  )dt 


t1 

[0)  = f eA  Vy(t,0)dt  . 


( 21-a ) 


( 21-b ) 


Specifically,  substituting  (16)  and  (17)  into  (18),  and  simplifying  the  resulting 


expression. 


An  iterative  solution  to  (19)  can  then  be  attempted  via  any  suitable  parameter 
search  method,  with  the  gradient  VJ^(0)  and  any  higher  order  derivatives  of 
J^(  0 ) computable  by  causal  relations  involving  the  input -output  data  on  [0,t  ], 
relative  to  any  particular  value  of  0 . Equation  (11-b)  must  be  integrated 
anew  (from  the  zero  state)  for  each  value  of  0 in  the  course  of  constructing 
a sequence  (0(n)}  as  a possible  solution  to  (19).  More  generally,  the  limit 
of  a minimizing  sequence  for  J^(0)  represents  a kind  of  least  squares 
••timate  of  the  parameter  vector  0 which  may,  or  may  not,  be  unique  depending 
on  the  nature  of  the  model  (1)  and  the  observed  data  on  TO,t^]  . 

In  the  case  of  a model  which  is  separable  in  the  parameters,  y(t,0)  is 
given  by  (12)  and  v(0)  in  (21-b)  can  be  written  as 


v(0)  = Nf (0 ) (22) 

where  the  matrix  N is  defined  by 

t1 

N = | eA,tC' M(t)dt  . .23) 

‘ o 

In  this  case,  the  function  J^(0)  reduces  to  the  following  explicitly  defined 
function  of  0 : 


J2(0)  = a - 2£'f(0)  + f'(0)4>f(0) 


(24) 


where  (djC,*)  are  defined  by 


. . f V 

'0 

C = M' 
* 


(t)h(t )dt  - n' w 1n 


(t)h(t )dt  - N'w-1n 


* = 


L _ 1 +■ 
M' (t  )M(t )dt  - N'W  N . 


( 25-a ) 


(25-b) 


(25-c) 


^he  matrix  ♦ is  actually  the  Gram  matrix  for  the  column  vector  functions  of 
M(t)  = {&(. M)  ; as  such,  $ is  necessarily  symmetric  and  non-negative  definite 
rega  dless  of  the  data  on  [0,t^]  . 
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Alt  hough  J1(6)  and  J2(8)  are  equivalent  positive  definite  functionals 
of  0 whose  zero  values  reflect  possible  values  for  the  unknown  parameter 
vector  0 satisfying  the  basic  model  (1),  the  computational  advantages  of 
(24)  for  the  separable-in-the-parameters  case  should  be  evident  inasmuch  as 
Jj(0)  is  an  explicitly  defined  function  of  6 , while  J^(0)  is  defined  only 
implicitly.  This  means  that  once  the  differential  equations  (11-a)  and  (13) 
are  integrated  (from  the  zero  state)  and  the  quantities  (n,N,o,C,$)  computed 
from  (21-a),  (23)  and  (25),  there  are  no  further  integrations  needed  involving 
the  data  collected  on  [0,t^]  . Moreover,  sufficient  conditions  for  the 
uniqueness  of  solutions  to  the  parameter  estimation  problem  can  be  stated  more 
specifically  as  follows. 

Assertion  In  the  case  of  the  basic  model  (1)  and  (2)  which  is  separable  in 

* 

the  parameters,  a minimizing  value  0 for  the  positive  definite  function 

J2(0)  in  (24)  is  a least  squares  estimate  of  the  parameter  vector  0 which  is 

unique  if  (as  a sufficient  condition)  the  given  data  is  such  that  the  columns 

of  Q(D)V(t)  are  linearly  independent  functions  on  0 i t ( t^  . 

* 

The  fact  that  a minimizing  value  0 for  J2(0)  is  a least  squares  esti- 
mate follows  from  the  consideration  that  if  J?(X)  , X = f(0)  is  minimized 
over  X , rather  than  0 , the  necessary  condition,  VJ^fX)  = 0 , is  seen  to  be 
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are  wholly  contained  in  the  subspace  (^,-  and  can  be  represented  by 
(F  ^(D)Q(D)V(t )X where  the  symbol  ( )X  denotes  the  orthogonal  complement, 
or  projection,  of  the  function  F ^(D)Q(D)V(t)X  in  the  subspace  (^  - ^ ) . 

Since  is  the  null  space  for  F(D)  , it  follows  that  linear  dependence,  or 

independence,  of  the  columns  of  M(»)  cannot  be  destroyed,  or  altered,  by 
operating  on  (F  ^(D)Q(D)V(t jX)1  with  F(D)  . By  this  argument,  the  sufficiency 
proof  for  uniqueness  in  the  Assertion  is  established.  This  condition  is  also 
necessary  when  the  function  f(0)  is  just  6 since  J2(0)  is  then  a positive 
definite  quadratic  form  in  0 . This  will  be  the  case  for  the  signal  estima- 
tion problem  in  Section  III,  as  well  as  any  problem  for  which  the  unknown  param- 
eters enter  linearly  in  the  basic  model  (1)  - (2). 
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In  the  model  (27-a)  for  the  signal  s(t)  which  is  to  be  estimated,  the  differ- 
ential operator  polynominals  (A(D),B(D))  and  the  deterministic  signal  u(t), 

0 s t * t^,  are  assumed  to  be  given,  but  the  initial  conditions  are  unknown.  The 
order  (r)  of  the  disturbance  model  (27-b)  is  assumed  to  be  specified,  but  the  co- 
efficients 9 = (6 ,**0  ) and  initial  conditions  are  unknown, 
l r 

Operating  on  both  sides  of  (26)  with  T(D,6)A(D),  noting  (27),  and  rearrang- 
ing terms  leads  to  the  differential  operator  equation 


0 = Dr[B(D)u(t)  - A(D)y(t)]  - [Dr-1-*-D  l][A(D)y(t)  - B(D)u(t)]  • (28) 


which  is  of  the  form  (1)  and  (2),  i.e.,  separable  in  the  parameters,  with  the 


parameter  vector  0 = Col(0^***8r)  entering  linearly.  A le  **st  squares  estimate 
of  0 is  obtained  either  by  minimizing  the  quadratic  functional  Jj(0)  with 
f(0)  = 0 in  (24),  or  by  solving  the  "normal"  equations 
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c = *0  (29) 

directly,  where  £ and  4>  are  given  in  (25)  after  first  integrating  the  differ- 
ential equations  (11-a)  and  (13)  for  this  problem,  i.e.,  obtaining  the  zero 
state  solutions  to 


F(D )h(t ) = DrB(D)u(t)  - DrA(D)y(t) 


( 30-a) 


F(D)M(t ) = [Dr-*  • 'D  l]fA(D)y(t)  - B(D)u(t)]. 


(30-b) 


Here  h(t)  is  a scalar  while  M(t)  is  a row  vector  (M, (t)**M  (t)). 

1 r 

Let  0 denote  the  resulting  least  squares  estimate  and  define  an  annihi- 
lation filter^-  by  its  impulse  response  function  H (t,T)  analogous  to  (14): 

0 “ 

At  A 1 T 

H0 (t , t ) = 6(t-T)-C0e  6 W^e  8 Cj  , 0 $ t,T  ? (31) 

where  (A.,C.)  is  an  observable  pair  for  the  disturbance  model  (27-b)  with 
0 0 

0=0,  e.g. 

f 0 1 0 I 


A-  = 00  •••  1 , C = Row(l,0«*0). 

0 e.  * “ 


The  signal  estimate  is  then  defined  by 


s(t)  =A/0(y(t)) 


o * t s t. 


i AfiT 

* 


- y(t)-C0e 


which  is  the  projection  of  the  observed  data  y(t)  down  into  the  subspace  ob- 
tained by  deleting  all  linear  combinations  of  the  disturbance  modes  identified 
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via  the  solution  to  the  normal  equations  (29).  Using  the  function  space  norm, 
the  error  in  this  estimate  can  be  found  to  have  the  following  representation: 


d'W.d 

060 


where 


■ “e1  f 1 

* r\ 


A 1 1 

e 9 C's(t)dt, 


It  is  clear  that  the  estimate  s(t)  is  precisely  the  signal  s(t)  under  the 
conditions:  (i)  the  models  in  (27)  are  correct,  (ii)  det  $ i 0,  and  (iii)  s(*) 
orthogonal  to  each  of  the  basis  functions  comprising  d(*),  i.e.  do  = 0. 

Example  2:  Fourier  Series  Model  for  s(t) 

Given  the  time  interval  [O.t^],  define  a fundamental  frequency  by 


U)  = 

o 


2it 


and  select  the  poljmomials  (A(D),B(D))  in  (27-a)  according  to 


A(D)  = D(D2+u)2)(D2+4W2).*(D2+m2u2) 
00  o 

B(D)  = 0. 


Then  the  solution  to  the  signal  model  (27-a)  is  the  truncated  Fourier  series 


m jnw  t 

s(t)  = l Sne  ° , 0 « t « tr 

n=-m 
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IV.  System  Identification 

A number  of  examples  will  be  given  in  this  section  to  illustrate  the 
basic  theory  for  system  parameter  identification. 

Example  3:  Linear  Differential  Systems 

Let  the  input-output  relation  for  a class  of  linear  systems  be  charac- 
terized in  the  absence  of  input  or  measurement  disturbances  by  the  differen- 
tial operator  equation 

A(D,w)y(t)  = B(D,w )u(t ) (34) 

n n 

where  A(D,w)  = £ A.(w)D  , B(D,w)  = £ B.(<o)Dn  . 

1=0  1=0 


The  coefficient  matrices  (A  ,.(<»))  ,B^(w))  ,0  $ i £ n,  are  assumed  to  be  given  func- 
tions of  a parameter  vector  u>  = (w , • *u>  ).  Define  a vector  valued  function  f(u>) 

1 P 

with  components  f^.(u)  selected  so  as  to  reflect  all  the  various  distinct  ways 
in  which  the  parameters  enter  into  the  A^(w)  and  B.(«),  i.e.  linear,  mvltipll- 
catively,  etc.  It  is  then  easy  to  see  how  to  define  quantities  (P(D),Q(D), 
v(t),V(t)>  with  v(t)  and  V(t)  depending  on  the  input-output  pair  Tu(t),y(t)] 
such  that  the  following  decomposition  holds: 


A(D,u>)y(t)  - B(D ,u)u(t)  = P(D)v(t)  - Q(D)V(t )f (u>) 

= 0. 


(35) 


Now  suppose  the  input  and  output  are  corrupted  by  additive  disturbances  such 
that  the  observed  output  y(t)  includes  a measurement  disturbance  d^(t)  accord- 
ing to 

y(t)  = yo(t)  + d^(t ) 

and  the  actual  input  to  the  system  is  the  signal 

u (t)  = u(t)  + d„(t ) 

O / 


j 


i 
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while  only  u(t)  can  be  directly  observed.  Including  these  disturbances  in  (34) 
implies : 

A(D,w)[y(t)  - d^t)]  = B(D,to)Cu(t)  + d?(t)]  . (36) 

If  d^(t)  and  d2(t)  are  assumed  to  be  arbitrary  solutions  to  differential  equa- 
tions of  the  form  (27-b)  on  the  observation  interval  [0,t^],  it  makes  no 
difference  whether  all  such  disturbance  modes  are  present  in  the  input,  that 
they  corrupt  only  the  output,  or  some  combination  thereof,  since  operating  on 
both  sides  of  (36)  with  T(D,0)  yields 

T(D,e)A(D,w)y(t)  = T(D,0)B(D,w)u(t).  (37) 

With  the  decomposition  (35)  already  defined  for  the  system  without  disturbances, 
the  analogous  decomposition  for  the  model  (37)  with  disturbances  modeled  by 


(27-b)  can  be  easily  shown  to  be  as  follows: 


0 = D P(D)v(t) 


-[Dr"1P(D)v(t)-*P(D)v(t):'D*'Q(D)V(t);Dx'-J-Q(D)V(t)**Q(D)V(t'>] 


k 


* r-1 

• n1  ±r 


0 


r 
f(w) 


e1f(u) 


e f(u>) 

r 


(38) 


This  differential  operator  equation  is  in  the  form  of  the  separable-in- 
the-parameters  model  (1)  and  (2)  with  parameter  vector  0 = (®i* ’©r’6*!* *wp) ’ 

The  vector  function  f(0)  on  the  right  side  of  (38)  satisfies  the  single-valued 
property  (3)  if  the  original  function  f(w)  in  (35)  satisfies  this  property. 

This  will  normally  be  the  case  for  a properly  parametrized  model. 

It  is  apparent  from  (37)  that  the  disturbances  can  be  equivalently  inter- 
preted as  uncontrollable  modes.  In  contrast  with  the  formulation  in  T5l,  the 

parameters  (0,**0  ) for  these  modes  are  modeled  explicitly  here,  rather  than 
1 r 


implicitly.  This  avoids  the  polynomial  factorization  discussed  in  [5]  when 
the  disturbances  are  modeled  implicitly. 


As  specific  illustration  of  this  notation  consider  the  following  state 
equations  for  a helicopter  in  longitudinal  motion  as  given  by  Narendra  and 
Tripathi  [6]: 


x(t)  = 


y(t) 


all 

®12 

al3  ®14 

b12 

a2l 

a22 

a23  a24 

x(t)  + 

© 

b22 

U31 

© 

a33  © 

b31 

b32 

_ 0 

0 

1 0 - 

_ 0 

o_ 

0 

o 6" 

x(t)  . 

L ° 

0 

0 t 

u(t) 


According  to  Narendra  and  Tripathi  the  (b^  ,a32 ,a3U^  entries  (designated  by 
(«,«,»>  above)  varv  significantly  over  the  airspeed  range  60-170  knots, 
while  the  remaining  entries  maintain  relatively  constant  values.  In  addition, 
the  vertical  velocity  (x^)  is  difficult  to  measure,  and  the  pitch  rate  (x3) 
might  entail  high  frequency  measurement  noise.  Hence,  horizontal  velocity  (x^) 
and  the  pitch  angle  (x^)  are  regarded  here  as  the  measurable  outputs,  together 
with  the  measurable  collective  (u^)  ahd  longitudinal  cyclic  (u^)  pitch  control 
input  variables,  and  w = (b^^a,^  ,a34 ) is  regarded  as  the  parameter  vector  for 
identification.  Eliminating  the  states  (x?,x3)  in  order  to  obtain  an  input- 
output  relation  of  the  form  (34),  and  rearranging  this  equation  into  the  requi- 
site form  (35),  the  quantities  (P,0,v,V,f)  are  found  to  be  as  follows: 

P(D)  = P P3  + P,D2  + P„P  + P„  , P = 

o 1 2 3 o 

'(t)  = 


0 10  0 
0 0 0 0 
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V(t)  = 


0 

0 


"a23y2(t)  _y2(t) 

y3(t)  a22y2(t) 

0 0 


0 

0 


y3(t)  ^ -a2iyjl(t)  - a24y2(t)  - b^t) 


The  (P  ,P.,P,)  matrices  depend  on  the  fixed  (a..,b..)  values. 

l/J  i]  i] 

Example  4 : Time  Lag  Systems 

Let  the  model  (34)  in  Example  3 be  modified  to  include  an  unknown  pure 
time  delay  a: 

A(D,w)y(t)  = B(D,w)u(t-a),  a ? at  ,,Q. 

max  v ov) ; 

It  is  clear  that  a decomposition  such  as  (35)  will  not  exist  in  this  instance 
so  that  the  time-delay  system  is  not  separable  in  the  delay  parameter.  How- 
ever, input -output  disturbances  similar  to  those  included  in  (36)  can  still 
be  incorporated  leading  to  (37)  with  u(t)  replaced  by  u(t-a).  The  basic  model 
(1)  will  then  apply  in  this  case  for  appropriately  defined  quantities  (P,0,v,g) 
with  the  parameter  vector  0 comprised  of  (6,w,a). 

As  a specific  illustration  of  the  notation  involved,  consider  a single 
input -single  output  system  with 

2 

A(D,w)  = D +u>^D+u>2  , B(D,w)  = 

T(D,0)  = D+0  . 

The  input-output  relation  (39)  with  disturbances  is  then 

(D2+u1D+w2)[y(t)-d1(t)]  = u>3[u(t-a)+d2(t-a)] . 

Using  a first  order  disturbance  model  for  both  d^(t)  and  d?(t),  the  above  is 
equivalent  to  the  following  differential  delay  operator  equation  (obtained 
after  some  rearrangement) : 


Although  a is  unknown,  its  value  is  assumed  to  be  bounded  by  a given  number 

a as  indicated, 
max 
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D3y(t)  + [D2 


D 1] 


(u1+6)y(t) 

(u2+«i8)y(t)  - WgU(t-a) 
o>2©y (t ) - (D^0u(t-«) 


0 


(40) 


which  is  of  the  form  (l)  with  parameter  vector  6 = (6  ,a) . 

Even  though  the  "state"  of  the  models  (39)  and  (40)  is  infinite  dimen- 
sional, the  basic  theory  of  Section  II  leading  to  the  functional  J^(6)  in  (20) 
is  still  valid.  However,  the  given  data  must  be  assumed  to  include  the  past 

input  u(t)  for  teT-a  ,0]  in  addition  to  the  input-output  pair  [u(t),y(t)l  on 

max 

ro,t1l.  This  will  ensure  the  existence  of  a solution  to  (19)  for  computational 
considerations . 

Example  5 : Hammerstein  Model 

Consider  the  scalar  system  with  observed  input-output  pair  Tu(t),y(t)] 
on  ro,t.]: 

k < 

A(D,w)fy(t)  + d^(t ) ] = B(D ,w )C  [ a.(u(t))  + d?(t)]  (41) 

i=l 

where  d^(t)  and  d0(t)  are  unknown  disturbances.  This  is  a Hammerstein  model 
[7]  with  output  disturbance  d^(t)  and  intermediate  input  disturbance  dj(t) 
entering  the  system  after  the  zero  memory  nonlinearity  characterized  by  the 
parameters  a = Assuming  the  disturbance  model  (?7-b)  for  d^(t)  and 

d?(t),  the  model  (41)  is  modified  to 

k 

T(D,6)A(D,o>)y(t)  = T(D,0)B(D,w)  £ a.(u(t))3.  (42) 

i=l  1 

It  is  evident  that  with  appropriate  definitions  of  the  quantities  (P,Q,v,V,f)4 
(42)  can  be  arranged  in  the  form  of  the  separable -in -parameters  model  (1) 
and  (2)  with  parameter  vector  0 = (0,w,a),  given  any  particular  (A(D,u>),B(D,w)) 
polynomials  and  integer  r in  (27-b). 


-20- 


Example  6 : Peak  Output  System 

Consider  a scalar  system  characterized  by  the  model 


A(D,u)tc(t)+d^(t)]  * B(D,w)r u(t)+d^(t)] 


(43-a) 


y(t ) = a - (c(t)) 


2 t 


(43-fc) 


where  d^(t)  and  d^ft)  are  again  disturbances  of  the  type  modeled  in  (27-b),  c(t) 


is  an  intermediate  output  signal  preceding  a parabolic  nonlinearity  with  unknown 
peak  value  a,  and  ru(t),y(t)]  is  the  observed  input-output  pair  on  [0,t^].  Incor- 
porating the  disturbance  model  (27-b)  into  (43-a)  and  replacing  c(t)  by 


±Ca-y(t)] 


1/2. 


1/2 


±T(I),0)A(D,w)[o-y(t)j  = T(D,e)B(D,w)u(t). 


(44) 


Given  polynomials  (A(D,o)),B(D,to))  and  an  inteper  r for  the  disturbance  model 
(27-b),  it  is  clear  that  (44)  can  be  arranged  into  the  form  of  the  nonseparable 
basic  model  (1)  with  parameter  vector  0 = (0,w,a).  For  example,  taking  the 


second  order  system  with  first  order  disturbances: 


A(D,u)  = D^+w^D+uj^,  B(D,o»)  = u>3,  T(D,0)  = D+0 , the  rearrangement  of  (44)  in  this 


case  leads  to 


0 = [D3  D?  D 1] 


±[a-y(t)] 


1/2 


±(0+w^ )[a-y(t )1 


1/2 


,1/? 


±(0to1+a)2)Ca-y(t)]  - u>3u(t) 


,1/2 


± 0u2[a-y(t)]  - 0«3u(t) 


(45) 


Here,  the  term  P(D)v(t)  in  (1)  is  absent  so  that  the  inner  product  terms  in- 
volving h(t)  and  n in  (20)  are  zero,  leaving 


Notice  that  a shift  in  the  parabola  according  to  y = a-(c-c*V  is  unnecessary 


since  the  disturbance  model  for  dJ(t)  includes  c*  as  a special  case. 


j 
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J2(0)  = ) 1 ru(t,e)]2dt  - v’(e)w'1v(e)  = o (46) 

' O 

where  0 = The  ambiguity  in  sign  for  the  term  ±(ct-y)^2  in  (44) 

and  (45)  cannot  be  resolved  and  may  cause  nonunioueness  in  finding  the  zeros  of 
(46)  even  if  the  input  is  sufficiently  active  to  excite  all  the  modes  in  the 
system  on  [0,t^],  In  any  case,  the  initial  guess  for  the  peak  output  should 
be  chosen  to  satisfy 

a(l)  > Max  y(t) 

0 ( t ( tj 

in  the  construction  of  any  sequence  (0(n)},  n = 1,2«* , directed  at  solving  (46). 


J 


V.  Computational  Considerations 


Assuming  the  basic  model  (l)  has  been  obtained  for  any  particular  appli 


;ary  to  choose  the  polynomial  matrix  F(D)  in  (8)  of  suffi. 


cation,  it  i 


ciently  high  order  so  that  (11-a)  and  (11-b),  or  (13),  can  be  integrated 


without  involving  derivatives  of  the  data  on  the  right  hand  side  of  these 


equations.  Apart  from  det  F(D)  ^ 0,  the  selection  of  F(D)  is  quite  unrestricted 


and  the  modes  of  F(D)  can  be  selected  as  either  stable  or  unstable  since  all 


computations  are  confined  to  the  finite  interval  [0,  t ].  In  this  selection  it 


is  possible  to  take  advantage  of  cancelling  various  modes  in  P(D)  and  Q(D) 


i.e.  cancellation  of  poles  and  zeros  in  F (D)[P(D),  Q(D)],  in  order  to  simplify 


the  computations  for  h(t),  p(t,8)  or  M(t).  However,  it  is  necessary  to  include 


any  such  cancelled  modes  in  the  computation  of  Ce  for  the  determination  of  W 


in  (14b)  and  in  the  computation  of  (n,v(6)  or  N)  in  (21)  and  (23).  In  this 


can  be  computed  offline  and 


connection  it  should  be  evident  that  Ce  and 


stored  for  subsequent  online  computations  as  input-output  data  is  presented 


Various  simplifications  in  integrating  the  reouired  differential  equations 


can  be  gleened  by  a careful  comparison  of  the  equations  involved.  For  example 


comparing  (30a)  and  (30b)  which  are  needed  to  obtain  the  normal  equations  (29) 


in  the  signal  estimation  problem,  it  is  evident  that  the  components  of  M(t)  can 


be  obtained  by  pure  integrations  of  h(t),  viz 


Similar  simplifications  in  computing  the  columns  of  M(t)  will  be  evident  for 
the  integrations  arising  from  the  basic  model  (38)  in  Example  3,  as  well  as  in 


the  other  example 
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Concerning  the  choice  in  the  order  (r)  of  the  disturbance  model  (27-h), 
it  could  be  argued  that  such  a choice  can  never  Le  made  with  any  degree  of 
certainty  since  disturbances  are,  by  nature,  unknown.  Two  comments  are  ap- 
propriate in  this  connection:  (i)  The  minimization  of  either  J^(0)  in  (20), 
or  J2(0)  in  (24),  represents  a kind  of  generalized  least  squares  estimate  of 
the  parameter  vector  0 appearing  in  the  basic  model  (1)  or  (l)-(2),  since 
(20)  and  (24)  stem  from  the  Hilbert  space  norm  of  the  projected  error  func- 
tion z(t,0)  in  (16).  (ii)  If  the  minimal  value  of  ^(0)  or  J 2 ( 9 ) exceeds  a 
certain  small  value  c indicating  inadequacy  of  the  model,  a natural  course  of 
action  is  to  change  the  value  of  r and  recompute  the  new  functional.  If 
J(G;r)  denotes  the  functional  corresponding  to  a cei'tain  value  of  r in  (27-b), 
a reasonable  procedure  to  follow  might  be  to  start  with  a large  value  of  r, 
say  rmax*  compute  the  necessary  quantities  for  r = r^^,  then  note  that  the 
analogous  quantities  for  r < r^^  can  be  obtained  by  pure  integrations  of  the 
former  in  many  cases.  To  illustrate  this  point,  consider  the  quantities  h(t)  = 
h(t;r)  and  M(t)  = M(t;r)  in  (30-a)  and  (30-h)  for  the  signal  estimation  problem. 

Assuming  F(D)  is  chosen  so  that  h(t;r  ) and  M(t;r  ) are  causally  related 

max  max 

to  the  given  data,  and  assuming  the  same  F(D)  for  r < r , it  is  seen  that 

h(t;r)  = Dr  r'!,iax  h(t;r  ) 

max 

which  is  an  (r  - r)  fold  integral  of  h(t;r  ).  This  consideration  coupled 
max  max 

with  the  observations  in  (47)  can  significantly  reduce  the  number  of  integra- 
tions when  considering  various  orders  for  the  disturbances  model  (27-b). 

As  a final  remark  concerning  the  minimization  of  Jo(0)  in  (24)  for  models 
which  are  separable  in  the  parameters,  a choice  basically  exists  between  mini- 
mizing over  X = f(0),  or  minimizing  directly  over  0,  in  view  of  the  single 
valued  property  (3).  The  advantage  of  the  former  is  that  J2  is  quadratic  in 


s r.r.'  ■ ...rrr. 
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X while  it  is  generally  highly  nonlinear  in  0.  However,  this  advantage  is 
offset  by  the  fact  that  the  dimension  of  X may  be  very  much  greater  than  0 
as  can  be  seen  from  the  various  separable  examples  presented  in  the  preced- 
ing, sections. 
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VI.  Concluding  Remarks 

The  formulations  of  the  signal  estimation  and  system  identification 
problems  presented  in  this  paper  use  models  for  the  disturbances  which  are 
deterministic  homogeneous  linear  differential  equations  of  preselected  or- 
der (r)  on  a finite  time  interval  [0,t^].  The  shorter  the  time  interval,  the 
more  realistic  will  be  the  assumption  that  the  disturbances  can  be  so  modeled. 
On  the  other  hand,  the  time  interval  must  be  long  enough  so  that  the  data  con- 
tains enough  information  to  reflect  the  values  of  the  parameters  to  be  deter- 
mined in  the  basic  model.  This  can  be  put  into  somewhat  more  specific  terms 
by  referring  to  particular  examples,  viz.  the  matrix  $ in  the  normal  equa- 
tions (29)  for  the  signal  estimation  problem  must  be  sufficiently  positive 
definite  to  yield  a unique  solution.  A larger  t ^ enhances  this  possibility 

The  examples  for  illustrating  the  basic  model  (1)  are  surely  not  exhaus- 
tive. The  Fourier  series  model  in  Example  2 is  a natural  model  for  the  signal 
estimation  problem;  vet  there  may  exist  applications  for  using  the  more  general 
model  of  (27-a).  The  nonlinear  examples  for  system  identification  referred  to 
tandem  interconnections  of  zero  memory  nonlinearities  and  linear  dynamic  sub- 
systems. However,  other  identification  problems  can  be  handled  such  as  the 
Duffing  equation  with  input  disturbance  d(t): 

v(t)  + u^yCtJvCt)  + w2v(t)  + u3y3(t)  = w4[u(t)  + d(t)]. 

In  differential  operator  notation,  this  can  be  written  as 

D2y(t)  + 1/2  w,D(y2(t))  + w2y(t)  + w3y3(t)  = (i^Tu ( t )+d( t ) ] 

# 2 

having  noted  that  vy  = l/2D(v  ).  Employing  the  disturbance  model  (27-b)  and 


since  $ is  a Gram  matrix  for  functions  in  the  space 


proceeding  as  in  the  previously  cited  examples  will  lead  to  a model  which  is 
separable  in  the  parameters.  The  Van  der  Pol  equation  can  be  similarly  handled. 
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Finally,  although  most  of  these  examples  have  involved  time  invariant  systems, 
it  should  be  clear  from  Example  1 that  certain  time  varying  systems  can  be 
placed  in  the  form  of  (1). 

Ho  examples  involving  numerical  calculations  have  been  given  in  this 
paper.  However,  a number  of  computer  simulations  have  been  carried  out  for 
linear  system  identification,  both  fixed  and  time  varying.  Some  of  these  are 
reported  in  Section  V of  [5],  while  others  are  contained  in  a recent  thesis  by 
Chin  [8],  The  results  of  the  latter,  which  includes  an  aerospace  adaptive  con- 
trol application  with  time  varying  parameters,  will  be  reported  elsewhere. 
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